21 research outputs found
Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes
Active classification, i.e., the sequential decision-making process aimed at
data acquisition for classification purposes, arises naturally in many
applications, including medical diagnosis, intrusion detection, and object
tracking. In this work, we study the problem of actively classifying dynamical
systems with a finite set of Markov decision process (MDP) models. We are
interested in finding strategies that actively interact with the dynamical
system, and observe its reactions so that the true model is determined
efficiently with high confidence. To this end, we present a decision-theoretic
framework based on partially observable Markov decision processes (POMDPs). The
proposed framework relies on assigning a classification belief (a probability
distribution) to each candidate MDP model. Given an initial belief, some
misclassification probabilities, a cost bound, and a finite time horizon, we
design POMDP strategies leading to classification decisions. We present two
different approaches to find such strategies. The first approach computes the
optimal strategy "exactly" using value iteration. To overcome the computational
complexity of finding exact solutions, the second approach is based on adaptive
sampling to approximate the optimal probability of reaching a classification
decision. We illustrate the proposed methodology using two examples from
medical diagnosis and intruder detection
Scheduling for Urban Air Mobility using Safe Learning
This work considers the scheduling problem for Urban Air Mobility (UAM)
vehicles travelling between origin-destination pairs with both hard and soft
trip deadlines. Each route is described by a discrete probability distribution
over trip completion times (or delay) and over inter-arrival times of requests
(or demand) for the route along with a fixed hard or soft deadline. Soft
deadlines carry a cost that is incurred when the deadline is missed. An online,
safe scheduler is developed that ensures that hard deadlines are never missed,
and that average cost of missing soft deadlines is minimized. The system is
modelled as a Markov Decision Process (MDP) and safe model-based learning is
used to find the probabilistic distributions over route delays and demand.
Monte Carlo Tree Search (MCTS) Earliest Deadline First (EDF) is used to safely
explore the learned models in an online fashion and develop a near-optimal
non-preemptive scheduling policy. These results are compared with Value
Iteration (VI) and MCTS (Random) scheduling solutions.Comment: In Proceedings FMAS2022 ASYDE2022, arXiv:2209.1318
Constrained Active Classification Using Partially Observable Markov Decision Processes
In this work, we study the problem of actively classifying the attributes of
dynamical systems characterized as a finite set of Markov decision process
(MDP) models. We are interested in finding strategies that actively interact
with the dynamical system and observe its reactions so that the attribute of
interest is classified efficiently with high confidence. We present a
decision-theoretic framework based on partially observable Markov decision
processes (POMDPs). The proposed framework relies on assigning a classification
belief (a probability distribution) to the attributes of interest. Given an
initial belief, confidence level over which a classification decision can be
made, a cost bound, safe belief sets, and a finite time horizon, we compute
POMDP strategies leading to classification decisions. We present two different
algorithms to compute such strategies. The first algorithm computes the optimal
strategy exactly by value iteration. To overcome the computational complexity
of computing the exact solutions, we propose a second algorithm is based on
adaptive sampling to approximate the optimal probability of reaching a
classification decision. We illustrate the proposed methodology using examples
from medical diagnosis and privacy-preserving advertising.Comment: arXiv admin note: substantial text overlap with arXiv:1810.0009
Cost-Bounded Active Classification Using Partially Observable Markov Decision Processes
Active classification, i.e., the sequential decision making process aimed at data acquisition for classification purposes, arises naturally in many applications, including medical diagnosis, intrusion detection, and object tracking. In this work, we study the problem of actively classifying dynamical systems with a finite set of Markov decision process (MDP) models. We are interested in finding strategies that actively interact with the dynamical system, and observe its reactions so that the true model is determined efficiently with high confidence. To this end, we present a decision-theoretic framework based on partially observable Markov decision processes (POMDPs). The proposed framework relies on assigning a classification belief (a probability distribution) to each candidate MDP model. Given an initial belief, some misclassification probabilities, a cost bound, and a finite time horizon, we design POMDP strategies leading to classification decisions. We present two different approaches to find such strategies. The first approach computes the optimal strategy “exactly” using value iteration. To overcome the computational complexity of finding exact solutions, the second approach is based on adaptive sampling to approximate the optimal probability of reaching a classification decision. We illustrate the proposed methodology using two examples from medical diagnosis and intruder detection